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Method of enhancing a program stream 



FIELD OF THE INVENTION 

The invention relates to a method of enhancing a content that comprises 
original program streams, by adjunction of at least one enhancement program stream 
associated with at least one of said original program streams. 

The invention also relates to a computer program comprising instructions for 
5 implementing such a method, and an authoring system intended for executing such a 
program. 

The invention also relates to a signal obtained by applying such a method, and 
to a data carrier storing such a signal. 

The invention also relates to a device for reading such a signal. 
10 The invention notably applies to contents that are compliant with the DVD 

Specifications for Read-Only Disc, part 3 Video Specifications, version 1.1, December 1997 
(called DVD specifications in the following of the description). 

For instance, it allows the enhancement of a video content so as to provide a 
high-definition video content. 

15 

BACKGROUND OF THE INVENTION 

In the DVD specification, audio, video, sub-picture, and navigation data 

elementary streams are multiplexed at the pack level to form a program stream (called VOB - 

Video Object - in the DVD specification). 
20 One solution to produce an enhanced content tiiat complies with the DVD 

specification is to multiplex enhancement elementary stream(s) with the above-mentioned 

elementary streams at the pack level. 

As the elementary streams are multiplexed at the pack level, it is not possible 

for the players to read only part of tiiem. 
25 Therefore, with such a solution, players will have to read all the data of an 

enhanced program stream in order to achieve de-multiplexing and recovery of the elementary 

streams. 
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It means that the enhanced program streams must comply with the transfer rate restrictions 
defined in the DVD specifications (table 5-2 for contiguous streams and annex K for 
interleaved streams) so as to be readable by any DVD compliant player. 

As a consequence the upper limit of 10,8Mbps (defined in table 5-2 of the 
DVD specifications) cannot be exceeded. 

One of the objects of the invention is to propose a solution allowing to exceed 
this upper limit while permitting reading by any DVD compliant player. 



SUMMARY OF THE INVENTION 

This is achieved with a method of enhancing a content as claimed in claims 1 
to 3, with a program comprising instructions for implementing such a method as claimed in 
claim 4, with an authoring tool as claimed m claim 5, with a data carrier as claimed in claims 
6 and 8, with a signal as claimed in claims 7 and 9, and with a device for reading a signal as 
claimed in claim 10. 

The invention applies to any content that comprises contiguous blocks 
containing a program stream and interleaved blocks containing interleaved program streams, 
wherein the program streams are divided into program units and are readable independently 
of one another, and the occurrence of consecutive interleaved blocks is forbidden. 

According to the invention: 

- each program unit of tiie enhancement program stream is associated with a program unit of 
the original program stream; 

- the enhancement program stream is interleaved in the content at the program xmit level such 
that enhancement program units are as close as possible to the original program unit they are 
associated with, and the occurrence of consecutive interleaved blocks is avoided; 

- and the original program units point to the enhancement program units they are associated 
with. 

The invention makes it possible to introduce the enhancement data to tiie 
content in the form of one or more enhancement program streams while avoiding the 
occurrence of consecutive interleaved blocks. 

As the enhancement program streams are interleaved with the original 
program streams at the level of the program units, it is possible for players to skip them when 
reading. 
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Therefore players may either read the original program stream(s) only, or both 
the original program stream(s) and the enhancement program stream(s). 

Consequently it is not necessary anymore to comply with the 10,8Mbps 
transfer rate upper limit Players that cannot achieve transfer rates higher than 10,8Mbps will 
5 still be able to retrieve the original content. More recent players that can achieve higher 
transfer rates will be able to retrieve both the original content and the enhancement content 

In other words, the invention renders it possible to exceed the 10,8Mbps upper 
limit while maintaining compatibility with existing players that cannot achieve higher 
transfer rates. 

10 The feature claimed in claim 2 is advantageous because it requires less 

memory and it simplifies the playback implementation. 

The feature of claim 3 is advantageous, especially when the enhancement 
program units are read before the associated original program units (and therefore have to be 
buffered until the associated original program imits are read), because it minimizes the 

15 quantity of data to be buffered. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other aspects of the invention are further described with reference 
to the following drawings: 
20 - Fig. 1 is a schematic representation of the structure of a contiguous block, 

- Fig. 2 is a schematic representation of the structure of an interleaved block, 

- Fig. 3 is a schematic representation of a program unit according to the DVD specifications, 

- Fig. 4 is a schematic representation of an example of enhanced content according to the 
invention, 

25 - Fig. 5 is a schematic representation of an enhancement of a contiguous block that leads to 
an occmrence of consecutive interleaved blocks, 

- Fig. 6 is a block diagram of a method of enhancing a content according to the invention, 

- Fig. 7 is a first example of application of the method according to the invention, 

- Fig. 8 is a second example of application of flie method according to the invention, 
30 - Fig. 9 is a block diagram of a reading device according to the invention, 

- Fig. 10 is a block diagram of an authoring system according to the invention. 
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DESCRIPTION OF A PREFERRED EMBODIMENT 

The embodiment that will be described below relates to the DVD 
specification. This is not restrictive. The invention also applies to other types of content 
having a similar contiguous/interleaved structure in which the occurrence of consecutive 
5 interleaved blocks must be avoided. 

In the DVD specification, program streams are called Video Objects (VOB) 
and program units are called Video Object Units (VOBU), 

According to the DVD specifications, contents comprise contiguous blocks 
and interleaved blocks. In a contiguous blocks Video Objects Units are stored in a contiguous 
10 manner. 

Fig. 1 gives an example of two contiguous blocks CBi and CBj. The 
contiguous block CBi comprises a Video Object VOBi containing M Video Object Units 
(i,l), . . (i, M). The contiguous block CBj comprises a Video Object VOBj containing P 
Video Object Units 0,1), (jJP). The Video Object Units of flie Video Objects VOBi and 

15 VOBj are stored consecutively one after the other. 

In an interleaved block, Video Objects are divided into areas called interleaved 
units (ILVU), and the Video Objects are interleaved at the level of the interleaved units. 
According to the DVD specifications, an interleaved unit may comprise one or more Video 
Object Units (called VOBU). In the following, for simplification purposes, we will consider 

20 that an interleaved unit comprises only one Video Object Unit. 

Fig. 2 gives an example of an interleaved block IL comprising two Video 
Objects VOBk and VOBI. The Video Objects VOBk and VOBI comprise M Video Object 
Units (k,x) and (l,x) with x=l to M respectively. The Video Object Units (l,x) are stored just 
behind the Video Object Units (k,x). 

25 In an interleaved block, the reading device reads data sequentially while 

skipping data that do not require reading. For instance, in Fig. 2, arrows J indicate that the 
reading device is skipping the Video Object Units (l,x). In practice Video Object Units of 
interleaved Video Objects comprise a pointer, called reading pointer, pointing to the next 
Video Object Units in the same Video Object so as to allow skipping fi-om one Video Object 

30 Unit to the next. Therefore, Video Objects can be read independently one firom the other. 

As indicated in paragraph K5 of the DVD specifications, the occurrence of 
consecutive interleaved blocks must be avoided in order to achieve a seamless playback of 
the content. It means that two interleaved blocks must be separated by at least one contiguous 
block. 
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Fig, 3 shows the structure of a Video Object Unit. As represented in Fig- 3, a 
Video Object Unit VOBU comprises multiplexed packs including navigation packs N, audio 
packs A, video packs V and sub-picture packs SP. 

The invention deals with enhancing a content by adding one or more 
enhancement Video Objects to the content. Each enhancement Video Object is associated 
with an original Video Object. During playback of the enhanced content both the original 
Video Object and the associated enhancement Video Object will have to be retrieved. To 
avoid slowness of the playback, the enhancement information has to be stored close enough 
to the original information it is associated with. This is achieved by interleaving the 
enhancement Video Object in the original block to which the associated original Video 
Object belongs. 

Contiguous original blocks are thus transformed into interleaved blocks. This 
is shown in Fig. 4. In Fig. 4 a contiguous block CB is represented which comprises one single 
Video Object split up into M original Video Objects Units Ui, . . ., Um. An enhancement 
Video Object Units H_Ui (i=l,. . ., M) is associated with each of the original Video Object 
Unit Ui, . . Um. The enhancement Video Object Units HJJi, . . HJUm are interleaved with 
the original Video Object Units Ui, . . Um. The resulting block IL is an interleaved block. 

If the contiguous original block is adjacent to an interleaved block in the 
original content, transforming a contiguous original block into an interleaved block will lead 
to the occurrence of consecutive interleaved blocks in the enhanced content. Figure 5 gives 
an exainple of such a situation. Fig. S shows a sequence SI of three blocks. It comprises an 
interleaved block ILl, followed by a contiguous block CB2, followed by an interleaved block 
IL3. After enhancement, this sequence is transformed into a sequence SI' that comprises 
three consecutive interleaved blocks ILl IL2' and IL3'. 

The invention proposes a method to enhance such a content through 
interleaving while avoiding occurrence of consecutive interleaved blocks. Fig. 6 is a block 
diagram representing the steps of a content enhancing method according to the invention. 
The original content is processed block by block. The algorithm described below with 
reference to Fig. 6 uses a three blocks sliding window [Bj-i; Bi; Bi+i] centred on the block to 
be currently processed (Bi). The &st block of the window (Bj-i) was processed in the 
previous iteration of the algorithm. The third block of the window (Bi+i) will be processed in 
the next iteration of the algorithm. The current iteration deals with the enhancement of the 
central block Bi. In Fig. 6, it is assumed that the origmal content comprises Q blocks (thus i 
varies from 1 to Q). For each iteration i of the algorithm, the follovmg steps are executed: 
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In step KO it is checked whether the original block Bi is to be enhanced. If the original block 
Bi is to be enhanced (arrow Y), then step Kl is executed. If the original block Bi is not to be 
enhanced (arrow N) then step Kll is executed. 

In step Kl it is checked whether the original block to be processed (Bi) is a contiguous block; 
5 if Bi is a contiguous block (Bi=CB), step K2 is executed; if Bi is an mterleaved block (Bi=IL), 
step K9 is executed. 

In step K2 it is checked whether interleaving the enhancement Video Object(s) into the 
original block Bi would lead to an occurrence of consecutive interleaved blocks in the 
content (this is the case when Bi-i and/or Bj+i are interleaved blocks); if the answer is yes 
10 (arrow Y), step K3 is executed; if the answer is no (arrow N), step K9 is executed. 
In step K3 the original block Bi is split into a main block MBi and: 

- one pre-separation block SBl, if Bi-i is an interleaved block, 

- and/or one post-separation block SB2, if Bi+i is an interleaved block. 
After step K3, step K4 is executed. 

15 In step K4 the enhancement is achieved by: 

- interleaving of the enhancement Video Object units (H_MBi) associated with the original 
Video Object Units of the main block MBi, if any, into the main block, thereby transforming 
the contiguous main block MBi into an interleaved block MBi'; 

- interleaving of the enhancement Video Object Units (H_SB 1 / H_SB2) associated with the 
20 original Video Object Units of a contiguoxxs separation block (SBl / SB2), if any, into an 

interleaved block that is adjacent to said contiguous separation block (which means either 
into ILl ' or into MBi* for H_SB 1, and either into MBi* or into IL2* for H_SB2), 

- leaving the separation blocks SBl / SB2 unchanged and using them as separations between 
the interleaved block MBi' and the preceding/ following interleaved blocks. 

25 In step K9 the enhancement Video Object(s) are interleaved into the original block Bi, 
thereby generating an interleaved block Bi*. 
Step KIO is executed after step K4 and after step K9. 

In step KIO a pointer PT_H is added in the original program units to point to the associated 
enhancement program unit. 
30 At step Kll. the index i is incremented, and the next iteration of the algorithm is started. 

It is to be understood that this method is applicable to the enhancement of one 
or more Video Objects of an original block. 

The method depicted in Fig. 6 will now be further explained by way of two 

examples. 
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In Fig. 7 it is assumed that in the sliding window Wl: 

- the first block Bm is an interleaved block (ILl) comprising two original Video objects 
IL1_1 and IL1_2, and the associated two enhancement Video Objects H_IL1_1 and 
H_IL1_2, said Video Objects being interleaved in the following order: 

5 IL1_1/H_IL1_1/IL1_2/H_IL1_2 ; 

- the second block Bi is a contiguous block (CB2); 

- the third block Bi+i is an interleaved block (IL3) comprising two original Video objects 
IL3_landIL3_2, 

According to the invention, the contiguous block CB2 is split into three contiguous blocks: a 
10 pre-separation block CB2a, a main block CB2b, and a post-separation block CB2c. 
CB2b is enhanced with H_CB2b. 

With a first raiplementation of the method of the mvention (PI), the enhancement Video 
Objects H_CB2a and H_CB2c are interleaved mto the interleaved block that precedes the 
contiguous separation blocks, which means that: 
15 - H_CB2a is interleaved in ILl, thereby generating a new interleaved block ILl ' which 
comprises the interleaved Video Objects ILl_l/H_ILl_l/ILl_2/H_ILl_2/H_CB2a, 

- H_CB2c is interleaved with CB2b and H_CB2b, thereby generating an interleaved block 
CB2' which comprises the interleaved Video Objects CB2b/H_CB2b/H_CB2c. 

The pre-separation block CB2a is inserted between the interleaved blocks TLV and CB2\ 
20 The post-separation block CB2c is appended after CB2', 

With a second implementation of the method of the invention (P2), the enhancement Video 
Objects H_CB2a and H_CB2c are interleaved into the interleaved block that follows the 
contiguous separation blocks, which means that: 

- the interleaved block ILl remains unchanged; 

25 - H_CB2a is interleaved with CB2b and H_CB2b, thereby generating an interleaved block 
CB2" which comprises the interieaved Video Objects H_CB2a/CB2b/H_CB2b. 

- H_CB2c is interleaved at the beginnmg of the block IL3, thereby generating a new 
interleaved block IL3' which comprises the interleaved Video Objects H_CB2c/IL3_l/IL3_2. 
In both cases the times sequence is maintained. 

30 In Fig. 8 it is assumed that in the sliding window W2: 

- the first block Bn is an interleaved block (ILl) comprising two original Video objects 
IL1_1 and IL1_2 and the associated two enhancement Video Objects H_IL1_1 and H_IL1_2, 
said Video Objects being interleaved in the following order 
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ILl_l/H_rLl J/IL1_2/H_IL1_2 ; 

- the second block Bi is a contiguous block (CB2); 

- the third block Bj+i is a contiguous block (CB3). 

According to the invention, the contiguoxxs block CB2 is split into two contiguous blocks: a 
5 pre-separation block CB2a and a main block CB2b. 
CB2b is enhanced with H_CB2b. 

With a first implementation of the method of the invention (Pr)» the enhancement Video 
Object H_CB2a is interleaved into the interleaved block that precedes the contiguous pre- 
separation block, which means that: 
10 - H_CB2a is interleaved in Bn, thereby generating a new interleaved block ILl ' which 
comprises the mterleaved Video Objects ILl_l/H_ILl_l/ILl_2/H_ILl_2/H_CB2a, 

- the interleaved block CB2* comprises the interleaved Video Objects CB2b/H_CB2b, 
The pre-separation block CB2a is inserted between the interleaved blocks ILl* and CB2\ 
The post-separation block CB2c is appended after CB2\ 

15 With a second implementation of the method of the invention (P2'), the enhancement Video 
Object H__CB2a is interleaved into the interleaved block that follows the contiguous pre- 
separation block, which means that: 

- the interleaved block ILl remains xmchanged; 

- H_CB2a is interleaved with CB2b and H_CB2b thereby generating an interleaved block 
20 CB2' • which comprises the interleaved Video Objects H_CB2a/CB2b/H_CB2b. 

The contiguous blocks are split at the level of the Video Object Units. 

The first implementation of the invention is advantageous because it requires 
less memory and it simplifies the playback implementation. In this first implementation, the 
enhancement data are stored ahead of the separation blocks they are associated with. 
25 Therefore it is advantageous to use separation blocks of minimal size. According to the DVD 
specifications, the separation blocks have to contain a Pre-unit (PREU) because they precede 
an interleaved block. The minimal size of a Pre-unit corresponds to 1,5 seconds of video (see 
paragraph 2.4.67 of the DVD specifications). 

A player 10 is represented in Fig. 9. The player of Fig. 9 is designed for 
30 reading a video signal 12 stored on a data carrier (a video disc 13 in this example). According 
to the DVD specifications, the video signal includes playback control data, called Navigation 
Data (N_data in figure 9) and data to be played back called Presentation Data (P^data in 
figure 9). The player 10 comprises a Navigation Manager 30 and a Presentation Engine 35. 
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The Presentation Engine 35 follows instructions 32 issued by the Navigation Manager 30 to 
play back the Presentation Data P_data from the disc 13, 

The Presentation Engine comprises an optical head 40 for reading the 
Presentation Data from the disc, and a processing imit 45 composed of software(s) and 
5 microprocessor(s) for processing the Presentation Data so as to output a signal to be 
displayed on a display 49. 

When retrieving a Video Object, traditional players do not take into account 
the enhancement pointers PT_H that may be contained in it 

Players according to the invention are specifically designed for reading the 
10 enhancement pointer PH_T contained in an original Video Object Unit so as to retrieve the 
associated enhancement Video Object Unit. The original Video Object Unit and the 
associated enhancement Video Object Unit are then processed together so as to deliver an 
enhanced signal to be displayed. These operations are implemented by means of a specific 
software executed by the processing \mit 45. 
15 Fig. 10 shows an authoring system according to the invention. The authoring 

system of Fig. 10 comprises software memory means 50 storing a software SFT comprising 
instructions for implementing an enhancement method according to the invention, content 
storing means 52 for storing an original content P_S and enhancement data H_P_S, and 
processing means 54 for executing the software SFT so as to enhance the original content 
20 P_S with the enhancement data H_P_S and deliver an enhanced content P_S'. 

With respect to the described enhancement method, authoring system, data 
carrier, signal, and reading device, modifications or improvements may be proposed without 
departing from the scope of the invention. The invention is thus not linuted to the examples 
provided. 

25 The word "comprising" does not exclude the presence of other elements or 

steps than those listed in the claims. 



